Tag: forecasting

Stock market forecasting with prophet

Posted on September 12, 2017 by Eric D. Brown, D.Sc.

In a previous post, I used stock market data to show how prophet detects changepoints in a signal (https://pythondata.com/forecasting-time-series-data-prophet-trend-changepoints/). After publishing that article, I’ve received a few questions asking how well (or poorly) prophet can forecast the stock market so I wanted to provide a quick write-up to look at stock market forecasting with prophet.

This article highlights using prophet for forecasting the markets. You can find a jupyter notebook with the full code used in this post here.

For this article, we’ll be using S&P 500 data from FRED. You can download this data into CSV format yourself or just grab a copy from the my github ‘examples’ directory here. let’s load our data and plot it.

market_df = pd.read_csv('../examples/SP500.csv', index_col='DATE', parse_dates=True)
market_df.plot()

with

Now, let’s run this data through prophet. Take a look at https://pythondata.com/forecasting-time-series-data-prophet-jupyter-notebook/ for more information on the basics of Prophet.

df = market_df.reset_index().rename(columns={'DATE':'ds', 'SP500':'y'})
df['y'] = np.log(df['y'])
model = Prophet()
model.fit(df);
future = model.make_future_dataframe(periods=365) #forecasting for 1 year from now.
forecast = model.predict(future)

And, let’s take a look at our forecast.

figure=model.plot(forecast)

With the data that we have, it is hard to see how good/bad the forecast (blue line) is compared to the actual data (black dots). Let’s take a look at the last 800 data points (~2 years) of forecast vs actual without looking at the future forecast (because we are just interested in getting a visual of the error between actual vs forecast).

two_years = forecast.set_index('ds').join(market_df)
two_years = two_years[['SP500', 'yhat', 'yhat_upper', 'yhat_lower' ]].dropna().tail(800)
two_years['yhat']=np.exp(two_years.yhat)
two_years['yhat_upper']=np.exp(two_years.yhat_upper)
two_years['yhat_lower']=np.exp(two_years.yhat_lower)
two_years[['SP500', 'yhat']].plot()

S&P 500 Forecast Plot - Last two years of actuals vs Forecast — S&P 500 Forecast Plot – Last two years of Actuals (orange) vs Forecast (blue – listed as yhat)

You can see from the above chart, our forecast follows the trend quite well but doesn’t seem to that great at catching the ‘volatility’ of the market. Don’t fret though…this may be a very good thing though for us if we are interested in ‘riding the trend’ rather than trying to catch peaks and dips perfectly.

Let’s take a look at a few measures of accuracy. First, we’ll look at a basic pandas dataframe “`describe“` function to see how thing slook then we’ll look at R-squared, Mean Squared Error (MSE) and Mean Absolute Error (MAE).

two_years_AE = (two_years.yhat - two_years.SP500)
print two_years_AE.describe()
count    800.000000
mean      -0.540173
std       47.568987
min     -141.265774
25%      -29.383549
50%       -1.548716
75%       25.878416
max      168.898459
dtype: float64

Those really aren’t bad numbers but they don’t really tell all of the story. Let’s take a look at a few more measures of accuracy.

Now, let’s look at R-squared using sklearn’s“`r2_score“` function:

r2_score(two_years.SP500, two_years.yhat)

We get a value of 0.91, which isn’t bad at all. I’ll take a 0.9 value in any first-go-round modeling approach.

Now, let’s look at mean squared error using sklearn’s“`mean_squared_error“` function:

mean_squared_error(two_years.SP500, two_years.yhat)

We get a value of 2260.27.

And there we have it…the real pointer to this modeling technique being a bit wonky.

An MSE of 2260.28 for a model that is trying to predict the S&P500 with values between 1900 and 2500 isn’t that good (remember…for MSE, closer to zero is better) if you are trying to predict exact changes and movements up/down.

Now, let’s look at the mean absolute error (MAE) using sklearn’s “`mean_absolute_error“` function. The MAE is the measurement of absolute error between two continuous variables and can give us a much better look at error rates than the standard mean.

mean_absolute_error(two_years.SP500, two_years.yhat)

For the MAE, we get 36.18

The MAE is continuing to tell us that the forecast by prophet isn’t ideal to use this forecast in trading.

Another way to look at the usefulness of this forecast is to plot the upper and lower confidence bands of the forecast against the actuals. You can do that by plotting yhat_upper and yhat_lower.

fig, ax1 = plt.subplots()
ax1.plot(two_years.SP500)
ax1.plot(two_years.yhat)
ax1.plot(two_years.yhat_upper, color='black',  linestyle=':', alpha=0.5)
ax1.plot(two_years.yhat_lower, color='black',  linestyle=':', alpha=0.5)
ax1.set_title('Actual S&P 500 (Orange) vs S&P 500 Forecasted Upper & Lower Confidence (Black)')
ax1.set_ylabel('Price')
ax1.set_xlabel('Date')

In the above chart, we can see the forecast (in blue) vs the actuals (in orange) with the upper and lower confidence bands in gray.

You can’t really tell anything quantifiable from this chart, but you can make a judgement on the value of the forecast. If you are trying to trade short-term (1 day to a few weeks) this forecast is almost useless but if you are investing with a timeframe of months to years, this forecast might provide some value to better understand the trend of the market and the forecasted trend.

Let’s go back and look at the actual forecast to see if it might tell us anything different than the forecast vs the actual data.

full_df = forecast.set_index('ds').join(market_df)
full_df['yhat']=np.exp(full_df['yhat'])
fig, ax1 = plt.subplots()
ax1.plot(full_df.SP500)
ax1.plot(full_df.yhat, color='black', linestyle=':')
ax1.fill_between(full_df.index, np.exp(full_df['yhat_upper']), np.exp(full_df['yhat_lower']), alpha=0.5, color='darkgray')
ax1.set_title('Actual S&P 500 (Orange) vs S&P 500 Forecasted (Black) with Confidence Bands')
ax1.set_ylabel('Price')
ax1.set_xlabel('Date')
L=ax1.legend() #get the legend
L.get_texts()[0].set_text('S&P 500 Actual') #change the legend text for 1st plot
L.get_texts()[1].set_text('S&P 5600 Forecasted') #change the legend text for 2nd plot

This chart is a bit easier to understand vs the default prophet chart (in my opinion at least). We can see throughout the history of the actuals vs forecast, that prophet does an OK job forecasting but has trouble with the areas when the market become very volatile.

Looking specifically at the future forecast, prophet is telling us that the market is going to continue rising and should be around 2750 at the end of the forecast period, with confidence bands stretching from 2000-ish to 4000-ish. If you show this forecast to any serious trader / investor, they’d quickly shrug it off as a terrible forecast. Anything that has a 2000 point confidence interval is worthless in the short- and long-term investing world.

That said, is there some value in prophet’s forecasting for the markets? Maybe. Perhaps a forecast looking only as a few days/weeks into the future would be much better than one that looks a year into the future. Maybe we can use the forecast on weekly or monthly data with better accuracy. Or…maybe we can use the forecast combined with other forecasts to make a better forecast. I may dig into that a bit more at some point in the future. Stay tuned.

Forecasting Time Series data with Prophet – Trend Changepoints

Posted on September 1, 2017September 1, 2017 by Eric D. Brown, D.Sc.

In these posts, I’ve been looking at using Prophet to forecast time series data at a monthly level using sales revenue data. In this post, I want to look at a very interesting aspect of Prophet (and time series analysis) that most people overlook – that of trend changepoints. This is the fourth in a series of posts about using Prophet to forecast time series data. The other parts can be found here:

Trend changepoint detection isn’t an easy thing to do. You could take the naive approach and just find local maxima and minima but those may or may not be changes in the overall trend of your signal. There are many different methods for changepoint detection (a good paper looking at four methods can be found here – Trend analysis and change point techniques: a survey) but thankfully Prophet does trend changepoint detection behind the scenes for us (and it does a pretty good job of it).

For most people / tasks, the automatic detection performed by Prophet is good enough, but it never hurts to know how to tweak Prophet in case it misses some changepoints.

I’ve uploaded a jupyter notebook here and the sample data that I’m using here. Rather than use the monthly sales data I’ve been using, i wanted to use something that has a bit more of a noticable trend so I grabbed the S&P 500 index from FRED.

The jupyter notebook has bit more detail about loading data and running prophet for this data, so I’ll just throw the commands here and you can jump over there to see more detail. These first steps are no different than the standard data loading / prep for prophet discussed in previous posts.

market_df = pd.read_csv('../examples/SP500.csv', index_col='DATE', parse_dates=True)
df = market_df.reset_index().rename(columns={'DATE':'ds', 'SP500':'y'})
df['y'] = np.log(df['y'])
#lets take a look at our data quickly
df.set_index('ds').y.plot()

Now, let’s run prophet. Again, this is no different than the steps we’ve taken in previous posts for prophet.

model = Prophet()
model.fit(df);
future = model.make_future_dataframe(periods=366)
forecast = model.predict(future)

Prophet has created our model and fit the data. It has also (behind the scenes) created some potential changepoints. We can access these changepoints with .changepoints. By default, Prophet adds 25 changepoints into the initial 80% of the data-set. The number of changepoints can be set by using the n_changepoints parameter when initializing prophet (e.g., model=Prophet(n_changepoints=30).

You can view the changepoints by typing the following:

model.changepoints

In addition to viewing the dates of the changepoints, we can also view a chart with changepoints added.

figure = model.plot(forecast)
for changepoint in model.changepoints:
    plt.axvline(changepoint,ls='--', lw=1)

S&P 500 Prophet Model with Changepoints Added (in oragen)

Taking a look at the possible changepoints (drawn in orange/red) in the above chart, we can see they fit pretty well with some of the highs and lows.

Prophet will also let us take a look at the magnitudes of these possible changepoints. You can look at this visualization with the following code:

deltas = model.params['delta'].mean(0)
fig = plt.figure(facecolor='w')
ax = fig.add_subplot(111)
ax.bar(range(len(deltas)), deltas)
ax.grid(True, which='major', c='gray', ls='-', lw=1, alpha=0.2)
ax.set_ylabel('Rate change')
ax.set_xlabel('Potential changepoint')
fig.tight_layout()

SP500 Prophel Model changepoint Magnitudes

We can see from the above chart, that there are quite a few of these changes points (found between 10 and 20 on the chart) that are very minimal in magnitude and are most likely to be ignored by prophet during forecasting be used in the forecasting.

Now, if we know where trends changed in the past, we can add these known changepoints into our dataframe for use by Prophet. For this data, I’m going to use the FRED website to find some of the low points and high points to use as trend changepoints. Note: In actuality, just because there is a low or high doesn’t mean its a real changepoint or trend change, but let’s assume it does.

m = Prophet(changepoints=['2009-03-09', '2010-07-02', '2011-09-26', '2012-03-20', '2010-04-06'])
forecast = m.fit(df).predict(future)
m.plot(forecast);

S&P500 Prophet Model with Manually Set Changepoints

We can see that by manually setting our changepoints (and only using a few points), we drastically changed the model compared to the model that prophet built for us using the automatic detection of changepoints. Unless you are very sure about your trend changepoints in the past, its probably good to keep the defaults that prophet provides.

Conclusion

Prophet’s use (and accessibility) of trend changepoints is wonderful, especially for those signals / datasets that have significant changes in trend during the lifetime of the signal. That said, unless you are certain about your changepoints, it might be best to let prophet do its thing automatically.

Note: Please don’t think that because prophet does an OK job of forecasting the SP500 chart historically in this example that you should use it to ‘predict’ the markets. The markets are awfully tough to forecast…I used this market data because I knew there were some very clear changepoints in the data.

Forecasting Time Series data with Prophet – Part 3

Posted on August 28, 2017August 28, 2017 by Eric D. Brown, D.Sc.

This is the third in a series of posts about using Prophet to forecast time series data. The other parts can be found here:

In those previous posts, I looked at forecasting monthly sales data 24 months into the future. In this post, I wanted to look at using the ‘holiday’ construct found within the Prophet library to try to better forecast around specific events. If we look at our sales data (you can find it here), there’s an obvious pattern each December. That pattern could be for a variety of reasons, but lets assume that its due to a promotion that is run every December. You can see the chart and pattern in the chart below.

sales data plot — Sales Data – Note the spike every December

Prophet allows you to build a holiday‘ dataframe and use that data in your modeling. For the purposes of this example, I’ll build my prophet holiday dataframe in the following manner:

promotions = pd.DataFrame({
  'holiday': 'december_promotion',
  'ds': pd.to_datetime(['2009-12-01', '2010-12-01', '2011-12-01', '2012-12-01',
                        '2013-12-01', '2014-12-01','2015-12-01']),
  'lower_window': 0,
  'upper_window': 0,
})

This promotions dataframe consisists of promotion dates for Dec in 2009 through 2015, The lower_window and upper_window values are set to zero to indicate that we don’t want prophet to consider any other months than the ones listed.

Now that I have my promotions dataframe ready to go, I’ll run through the modeling quickly (you can check out the jupyter notebook for more details):

sales_df = pd.read_csv('../examples/retail_sales.csv', index_col='date', parse_dates=True)
df = sales_df.reset_index()
df=df.rename(columns={'date':'ds', 'sales':'y'})
df['y'] = np.log(df['y'])
model = Prophet(holidays=promotions)
model.fit(df);
future = model.make_future_dataframe(periods=24, freq = 'm')
forecast = model.predict(future)
model.plot(forecast);

With these steps, we’ve loaded the data, set it up the way prophet expects and ran our model with the promotions data and then plotted the model, which looks like the following:

Given that we have such little data, I doubt the use of holidays will make that much difference in the forecasts, but its a good example to use. We can check the difference in the model with holidays vs the model without by re-running the prophet forecast without holidays and see that the average difference between the two is ~ 0.06%…which isn’t terribly large, but still worth investigating. The jupyter notebook that accompanies this post goes into much more detail on this aspect (as well as the overall analysis).

Note: You can find the full code for this post in a Jupyter notebook here:

Jupyter Notebook for Forecasting Time Series data with Prophet – Part 3