Hi,
Following http://www.dynare.org/phpBB3/posting.php?mode=reply http://www.dynare.org/phpBB3/posting.php?mode=reply&f=1&t=7287 &f=1&t=7287, what would be the easiest way to do out of sample forecasting, i.e. use say observations 1 to 100 to estimate the model and the run a forecasting exercise on the next 100 observations? The calib_smoother command implicitly has to capability via the filtered_variables, but this would allow point-forecasts. Is there a better way? If not, we should maybe provide the capacities for this.
Best,
Johannes
I'm not sure that I understand the request. Is it a tabular version of the intervals that we draw around the forecast, taking into account the posterior distribution of the parameters? I don't think there is meaningful synthetic statistics because the uncertainty surounding the forecast depends of the horizon.
Best
Michel
Johannes Pfeifer writes:
Hi,
Following http://www.dynare.org/phpBB3/posting.php?mode=reply http://www.dynare.org/phpBB3/posting.php?mode=reply&f=1&t=7287 &f=1&t=7287, what would be the easiest way to do out of sample forecasting, i.e. use say observations 1 to 100 to estimate the model and the run a forecasting exercise on the next 100 observations? The calib_smoother command implicitly has to capability via the filtered_variables, but this would allow point-forecasts. Is there a better way? If not, we should maybe provide the capacities for this.
Best,
Johannes
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
I am not necessarily talking about the request. This was a more general issue. I was just wondering how one can do out-of-sample forecasting with forecasts of any horizon. Currently, Dynare only supports in-sample forecasts (with the exception of the end of the sample). Say I have 200 observations. I want to use the first 100 observations for estimating the model parameters and the last 100 observations for testing the model using out-of-sample forecasts. Denote the forecast horizon with k. For each observation t=101:200 I want the k-period ahead forecasts based on estimated parameters for t=1:100. That means, I need to first estimate the model on this fixed sample and then, given the parameters, run the smoother to extract the states, and then get the forecasts based on the parameters and the state estimates. The evaluate_smoother command only gives a point forecast that does not provide any measure of parameter and shock uncertainty. Recursive estimation would provide such out-of-sample forecasts for a range of observations, but when considering observation t+1, it will reestimate the parameters on this sample. But I would like to keep the sample fixed.
Something like this would allow the user to answer her question. For each horizon k, one could compute the RMSE over the 100-k forecasts made.
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Montag, 14. September 2015 20:18 An: List for Dynare developers Betreff: Re: [DynareDev] Out of sample forecasting
I'm not sure that I understand the request. Is it a tabular version of the intervals that we draw around the forecast, taking into account the posterior distribution of the parameters? I don't think there is meaningful synthetic statistics because the uncertainty surounding the forecast depends of the horizon.
Best
Michel
Johannes Pfeifer writes:
Hi,
Following http://www.dynare.org/phpBB3/posting.php?mode=reply http://www.dynare.org/phpBB3/posting.php?mode=reply&f=1&t=7287 &f=1&t=7287, what would be the easiest way to do out of sample forecasting, i.e. use say observations 1 to 100 to estimate the model and the run a forecasting exercise on the next 100 observations? The calib_smoother command implicitly has to capability via the filtered_variables, but this would allow point-forecasts. Is there a better way? If not, we should maybe provide the capacities for this.
Best,
Johannes
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-- Michel Juillard _______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Isn't it what the forecast option of estimation() does?
RMSE would imply several forecasts at k horizon.
Johannes Pfeifer writes:
I am not necessarily talking about the request. This was a more general issue. I was just wondering how one can do out-of-sample forecasting with forecasts of any horizon. Currently, Dynare only supports in-sample forecasts (with the exception of the end of the sample). Say I have 200 observations. I want to use the first 100 observations for estimating the model parameters and the last 100 observations for testing the model using out-of-sample forecasts. Denote the forecast horizon with k. For each observation t=101:200 I want the k-period ahead forecasts based on estimated parameters for t=1:100. That means, I need to first estimate the model on this fixed sample and then, given the parameters, run the smoother to extract the states, and then get the forecasts based on the parameters and the state estimates. The evaluate_smoother command only gives a point forecast that does not provide any measure of parameter and shock uncertainty. Recursive estimation would provide such out-of-sample forecasts for a range of observations, but when considering observation t+1, it will reestimate the parameters on this sample. But I would like to keep the sample fixed.
Something like this would allow the user to answer her question. For each horizon k, one could compute the RMSE over the 100-k forecasts made.
-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Montag, 14. September 2015 20:18 An: List for Dynare developers Betreff: Re: [DynareDev] Out of sample forecasting
I'm not sure that I understand the request. Is it a tabular version of the intervals that we draw around the forecast, taking into account the posterior distribution of the parameters? I don't think there is meaningful synthetic statistics because the uncertainty surounding the forecast depends of the horizon.
Best
Michel
Johannes Pfeifer writes:
Hi,
Following http://www.dynare.org/phpBB3/posting.php?mode=reply http://www.dynare.org/phpBB3/posting.php?mode=reply&f=1&t=7287 &f=1&t=7287, what would be the easiest way to do out of sample forecasting, i.e. use say observations 1 to 100 to estimate the model and the run a forecasting exercise on the next 100 observations? The calib_smoother command implicitly has to capability via the filtered_variables, but this would allow point-forecasts. Is there a better way? If not, we should maybe provide the capacities for this.
Best,
Johannes
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
Isn't it what the forecast option of estimation() does?
No. If the model is estimated at the first 100 observations (nobs=100), the forecast option does the last forecast at observation 100, i.e. E_100(y_100+k). But I want to have E_101(y_101+k), E_102(y_102+k), E_103(y_103+k),...m E_200(y_200+k)
RMSE would imply several forecasts at k horizon.
Exactly. But this is what I would get in the case outlined above. 100 k-step ahead forecasts made conditional on information available at t=101,...,t=200.
OK, I got it, now. I think we call these pseudo out of sample, because we do have observations for them.
We now have them with recursive estimation, but we should add it for fixed calibrated/estimated-once parameters and comute RMSEs
Johannes Pfeifer writes:
Isn't it what the forecast option of estimation() does?
No. If the model is estimated at the first 100 observations (nobs=100), the forecast option does the last forecast at observation 100, i.e. E_100(y_100+k). But I want to have E_101(y_101+k), E_102(y_102+k), E_103(y_103+k),...m E_200(y_200+k)
RMSE would imply several forecasts at k horizon.
Exactly. But this is what I would get in the case outlined above. 100 k-step ahead forecasts made conditional on information available at t=101,...,t=200.
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
I will open a ticket. But recursive estimation does not allow for what I described as the information set for the estimation of parameters also expands
Am 15.09.2015 um 08:05 schrieb Michel Juillard:
OK, I got it, now. I think we call these pseudo out of sample, because we do have observations for them.
We now have them with recursive estimation, but we should add it for fixed calibrated/estimated-once parameters and comute RMSEs
Johannes Pfeifer writes:
Isn't it what the forecast option of estimation() does?
No. If the model is estimated at the first 100 observations (nobs=100), the forecast option does the last forecast at observation 100, i.e. E_100(y_100+k). But I want to have E_101(y_101+k), E_102(y_102+k), E_103(y_103+k),...m E_200(y_200+k)
RMSE would imply several forecasts at k horizon.
Exactly. But this is what I would get in the case outlined above. 100 k-step ahead forecasts made conditional on information available at t=101,...,t=200.
Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev
-
Johannes Pfeifer -
Michel Juillard