Journal of Data Science

Abstract: Relative entropy identities yield basic decompositions of cat egorical data log-likelihoods. These naturally lead to the development of information models in contrast to the hierarchical log-linear models. A recent study by the authors clarified the principal difference in the data likelihood analysis between the two model types. The proposed scheme of log-likelihood decomposition introduces a prototype of linear information models, with which a basic scheme of model selection can be formulated accordingly. Empirical studies with high-way contingency tables are exem plified to illustrate the natural selections of information models in contrast to hierarchical log-linear models.