A Simple and General Exponential Family Model for Partial Membership and Factor Analysis
Shakir Mohamed, Katherine A. Heller and Zoubin Ghahramani
Description
This code provides implementations of unsupervised model with latent/hidden variables for the analysis of non-Gaussian data. Implementations of two models are provided: the Bayesian Partial Membership Model (BPM), and Exponential Family Factor Analysis/PCA (EXFA). These models allow for a wide range of statistical and modelling tasks, including:
- Visualisation of high-dimensional data,
- Dimensionality reduction,
- Missing data imputation,
- Analysis of heterogeneous (mixed) data sets.
Contents
Matab code for Hybrid (Hamiltonian) Monte Carlo Sampling for exponential family PCA and Bayesian Partial Membership (see list of related papers below).
Download the Code
This code was written and tested using Matlab 2012a (7.14).
- 25 February 2013: Initial release. Download EFLVM.zip
Setup
Add all paths to Matlab and specify the directories to be used.
% Start matlab in the main directory and add: addpath(genpath(pwd)); % On inital use change the directories that are to be used in the file: disp('change folders in setup.dir') edit setupDir.m disp('paused .') pause
Demo examples
1. Example of use of HMC for Bayesian logistic regression. Use link below for details and results.
demo_BayesLogReg;
2. Examples on synthetic data for EFA and BPM. Code provides analysis of Binary data only at present. Extensions to other cases follow similarly to the binary case. Use link below for details and results.
demo_EFA; demo_BPM;
Experiments in chapter
All experiments and figures appearing in the chapter can be reproduced.
plotEFLVM('synth-efa'); plotEFLVM('synth-bpm'); plotEFLVM('votes-bpm'); plotEFLVM('votes-efa'); plotEFLVM('fuzzykm');
Key papers related this code:
- Shakir Mohamed, Katherine A. Heller and Zoubin Ghahramani, A Simple and General Exponential Family Model for Partial Membership and Factor Analysis, chapter in Handbook of Mixed Membership Models. 2013.
- Shakir Mohamed, Katherine A. Heller and Zoubin Ghahramani. Bayesian Exponential Family PCA, in NIPS 2008.
- Katherine A. Heller, Sinead A. Williamson and Zoubin Ghahramani, Statistical Models for Partial Membership, in ICML 2008.
Support Packages
We make use of Matlab and the statistics toolbox, the lightspeed package and other utilities. These additional files are included in the folder support/.
Miscellaneous
This code may have bugs. If you find any errors, please let us know.
Shakir