Approach for highly unbalanced panel dataset

I have some highly unbalanced panel data set for which I want to construct a predictive model. As the response variable consists of count data I chose to use a Negative Binomial regression model. Due to possible multiple number of observations per individual over time I would expect the observations per individual to be possibly … Read more

What test should I use with a dichotomous dependent variable and a continuous independent variable for agreement analysis?

I’m conducting a study in which I’m testing interrater agreement on a new radiologic technique to the pathological evaluation (gold standard). I’ve got 58 patients and 15 raters. Raters have been asked to evaluate the degree of cancer invasion (‘Deep’ vs. ‘Superficial’) and have also reported how certain they are in their assessment, using a … Read more

Log and inverse Data transformation in Linear regression model

I am studying behaviour of particulate matters(pm10) concentration in respose to change in rain and tempretaure. my data was not normaly distributed so i have to transform data I did log transformation and inverse transformation.The Adjusted R-squared for log transformation is :0.07918 and Adjusted R-squared for inverse transform is :0.1002.Now according to rule i must … Read more

Independence of points from same underlying function

I have a function y=sin(x) I sample the points xi={0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1} and yi=sin(xi)+ϵ  ϵ∼iid Normal Now given only the set {xi,yi} I want to uncover the underlying functional form f(x) given as y=f(x)+ϵ    ϵ∼iid Normal My question is : ARE yi‘s independent ? My intuition is that y1 and y2 are not independent. This is because given a new x … Read more

Residuals correlated positively with response variable strongly in linear regression

I did the multiple linear regression on a dataset of 412 observations, with one response variable (Y) and 25 explanatory variables(X1-X25). Y and most of Xs are not normally distributed. Besides, there are some correlation between several Xs. The plot show that the residuals strongly correlated with Y positively and weakly correlated with fitted Y … Read more

Decorrelating Systems of Random Variables

Suppose we have a dataset that includes dozens of attributes, that are all correlated with each-other. I would like to better understand which variables affect which other variables in a causal way, preferably as a system of regressions. The classical approach would be to setup a structured model, identifying manually which attributes are endogenous and … Read more

Estimating burstiness in a time series: Dependence on overall frequency?

I have a dataset of time series normalized for length. A working R example below. I am trying to estimate burstiness within the series, i.e. to compare series in the degree of clusteredness in these events. Here I just used a simple index of dispersion (variance of intervals divided by mean of intervals). For this … Read more

correlation matrix test: is this code correct or is it missing a multiple comparisons correction?

I have $m$ variables $x_1,\dots,x_m$, measured in $N$ independent tests $\{x_{i1},\dots,x_{im}\}_{i=1}^N$, leading to the design matrix $X$. I noted that the demo function corrplot_intro.r from the R package corrplot includes a nice function cor.mtest (reported below), which computes the pairwise correlations for $x_1,\dots,x_m$ in the sample $X$ and reports the corresponding $p$-values: cor.mtest <- function(mat, … Read more

multiple testing correction for pairwise correlations

I have a dataset comprised of two $n$ x $d$ matrices, $A$ and $B$. Each is comprised of $n$ observations of $d$ angles. The angles are taken from two different meshes of planar triangles that are in correspondence such that columns $A_i$ and $B_i$ both represent $n$ measurements of angle $\alpha_i$. Many columns of $A$ … Read more

How to compute the correlation between a qualitative and a quantitative variable? [duplicate]

This question already has answers here: Correlations with unordered categorical variables (6 answers) What is Polychoric Correlation Coefficient intuitively? (1 answer) Closed 5 years ago. My study is about the correlation of the person’s gender and his or her score in a questionnaire about gender stereotyping. The higher the score in the questionnaire means he … Read more