I need to estimate a logistic regression that is penalized by the L1-norm. Choosing the penalty parameter by cross-validation is too computationally expensive for my application. I was wondering whether there was a simple formula I can use? I am aware that simple formulas exist in the linear case but do not know of one … Read more
Suppose we have a dataset that includes dozens of attributes, that are all correlated with each-other. I would like to better understand which variables affect which other variables in a causal way, preferably as a system of regressions. The classical approach would be to setup a structured model, identifying manually which attributes are endogenous and … Read more
Could lasso be used by itself as a feature selection method without being tied to a regression loss function? And if so, what’s a good package for doing so? Also, could lasso be performed as a step before using a classifier like random forest? Answer AttributionSource : Link , Question Author : casualprogrammer , Answer … Read more
In my data set I have only 6 observations & independent variables are 9. Can I use Lasso regression or multiple regression in this situation? when independent variables are > observations? Data : y x1 x2 x3 x4 x5 x6 x7 x8 x9 6142.8 90.25 164.19 15 0.91 0.88 2.99 0.5 7.255 15 8174.2 126.9 … Read more
I have 8 samples with outcome 0 or 1 (4 samples of each) and I have 3100 variables, is it possible to run elastic net on these using an alpha of 1 (I just want to use lasso)? And is it possible to do in R? Answer AttributionSource : Link , Question Author : user3324491 … Read more
This is my first post here – I’m not a statistician by training though I have a machine learning background, so please correct any erroneous usage of statistical terminology if you see any. Currently, I have a (linear) regression problem I’m trying to solve – ~5000 predictors and ~200 examples which is an extremely ill-posed … Read more
As proved by Zou (2006), the adaptive LASSO enjoys the Oracle Property in the sense that it does variable selection as if it knew the true (sparse) model. However, in the early days, the standard hypothesis testing in e.g. OLS should also point in the direction of which variables to leave out in the model … Read more
All those online courses that mention logistic regression with regularization explain the L2 regularization. This makes sense because L2 has a derivation-based analytical solution, which leads to, in updating the coefficients, just reducing them by a factor (proportional to the penalty parameter), and it is very intuitive. However, I need to figure out how to … Read more
I have $n$ observations of $p$ random variables $\{Y_1,\dots,Y_p \}$. I know that each observation has been generated from a linear network model of the form $$ Y = WY+\epsilon $$ and that the graph described by $W$ is likely to be sparse and to have cycles. What is the current state of the art … Read more
Lasso uses cross-validation to determine both the number of included predictors and the degree of shrinkage to avoid over-fitting. I have used the glmnet package to do this. fit=glmnet(x,y) cvob1= cv.glmnet(x,y) lambda1=cvob1$lambda.min cvob2=cv.glmnet(x,y) lambda2=cvob2$lambda.min There is a high possibility that lambda1 and lambda2 are different. I need to get a proper lambda.min for the next … Read more