How to tell if girlfriend can tell the future (i.e. predict stocks)?

My girlfriend has recently gotten a job doing sales and trading at a major bank. Buoyed by her new job, she believes she can predict whether stocks will be up or down at the end of the month greater than chance (she believes she can even do it with 80% accuracy!)

I am very skeptical. We have agreed to do an experiment in which she will choose a number of stocks and, at a predetermined time, we will check if they are up or down.

My question is this: how many stocks would she have to pick, and how many would she have to get right, in order to have enough statistical power to tell with confidence that she can accurately predict stocks?

For example, how many stocks would she have to pick to tell with 95% certainty that she pick stocks with 80% accuracy?

For the experiment we agreed to, she does not have to predict by how much stocks will be up or down, but only if they will be up or down.


Interesting question. This isn’t really an answer, but it’s too long to be a comment.

I think your experimental design is challenged for these reasons:

1) This does not reflect the way that stock picking is actually evaluated in the “real world”. As an extreme example, suppose stock picker A chose 1 stock that went up 1000%, and 9 that went down by 1%, and stock picker B chose 10 stocks that all went up 1%. If these stocks were actually used to construct an index, then clearly A would be the better performer, but B would do much better in your experiment. A more financially interesting challenge would be to construct a portfolio and compare its performance to that of the S&P 500. In turn, there is a commonly-used machinery for evaluating such performance: simply take a linear regression of the day-to-day returns of the portfolio against those of the S&P. The intercept term (often called “alpha”) measures the average performance “over and above the market”. Since it is a coefficient of a linear regression, it is a trivial matter to construct a 95% confidence interval if you so choose. Then compare this to the fees her bank would charge for this service.

2) Disregarding 1, since it sounds like you both have already agreed on the form the experiment, consider how this could be gamed. Suppose I had a magic oracle that told me the probability of each stock being above its current price a month from now (say). Then I could just pick the n stocks with the highest such probabilities, and most likely over 50% of them would indeed go up. Now, such probabilities are encoded (imperfectly) in various options prices. For example, I can buy a so-called “binary option”, which is basically just a gamble on the event “Stock X willl be above price Y on date Z”. The pricing of such implies a probability of this event (although the closer date Z is to the present, the less reliable this will be). Since blindly following the “wisdom of the crowds” requires no particular expertise, I would argue that the performance of a strategy like this should be considered “chance levels” for your particular experiment. Alternatively, you present her with a list of stocks of your choosing, and have her indicate whether she thinks each will be up or down, together with her confidence on each prediction. Then group all answers by confidence level and see how closely they align (i.e., of those stocks that she was 90% confident about, did she correctly predict 90% of them?). There’s a standard way to quantify this; i don’t remember offhand what it’s called, but you can read about it in Superforecasters by Phil Tetlock.

Source : Link , Question Author : Benjamin , Answer Author : Mike Hawk

Leave a Comment