ANALYSIS OF TEXT NON-HOMOGENEITY USING MARKERS

Monika Lapėnaitė-Gedvilė, Karolina Piaseckienė, Marijus Radavičius

Abstract


The aim of the paper is to assess the distributional non-homogeneity of texts in the usage of functional words andother linguistic units. Our empirical study is based on recommended school fiction works taken from a digital library athttp://ebiblioteka.mkp.emokykla.lt. Sets of frequent word forms, called markers, are made, and their frequency counts in blocks of 50successive sentences are calculated. The frequency counts of the markers show significant excess variability (overdispersion) withrespect to a text homogeneity model usually assumed in linguistics. For chosen markers, different kinds of hierarchical binomiallogistic regression models with the author's identifier, the block length and the frequency counts of the remaining markers asexplanatory variables are fitted to the block data in order to explain the observed overdispersion of the markers chosen.

Keywords


statistical linguistics, over-dispersion, deviance, binomial logistic regression, functional words.

Full Text: PDF

Lithuanian Journal of Statistics ISSN 1392-642X, eISSN 2029-7262
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 License.