Hi,

it's not a problem with the testsuite, but with Matlab's convention that you can subtract a conformable vector from a matrix, because Matlab expands the vector to a matrix. Simply try to run

eye(4)-[1 2 3 4]

and you will see that it returns

ans =

0 -2 -3 -4 -1 -1 -3 -4 -1 -2 -2 -4 -1 -2 -3 -3

That's why no error was triggered here. Problems like these are very hard to detect, unless you know the correct answer of the computations and check for them. This might be a next step.

Best, Johannes

-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Samstag, 19. November 2016 10:20 An: List for Dynare developers Betreff: [DynareDev] Testsuite reports erroneous results

I don't undestand how the testsuite can pass test ./kalman_filter_smoother/algo4.mod and several other test cases because of issue https://github.com/DynareTeam/dynare/issues/1339 This test fails on my machine

If the testsuite isn't reliable, we have a major problem

Best

Michel

Dynare Robot writes on 18/11/2016:

2f9dc092855bbdb0b3fdd970b6ab551842c5080a diffuse filter: removed special treatment of static variables ================================ DYNARE MAKE CHECK MATLAB RESULTS ================================ | TOTAL: 705 | PASS: 697 | FAIL: 1 | XFAIL: 7 | XPASS: 0 | LIST OF FAILED TESTS: | * arima/mod2a.mod | | LIST OF 10 SLOWEST TESTS: | * optimizers/fs2000_6.mod - 739.560000 | *

observation_trends_and_prefiltering/MCMC/Trend_loglinear_no_prefilter_MC.mod - 435.730000

| *

observation_trends_and_prefiltering/MCMC/Trend_no_prefilter_first_obs_MC.mod - 434.400000

| *

observation_trends_and_prefiltering/MCMC/Trend_loglin_no_prefilt_first_obs_M C.mod - 399.260000

| * observation_trends_and_prefiltering/MCMC/Trend_no_prefilter_MC.mod

- 396.370000

| *

observation_trends_and_prefiltering/MCMC/Trend_loglin_prefilt_first_obs_MC.m od - 376.320000

| * observation_trends_and_prefiltering/MCMC/Trend_prefilter_MC.mod -

372.550000

| * ep/rbc_mc.mod - 371.740000 | * gsa/ls2003.mod - 370.280000 | * estimation/TaRB/fs2000_tarb.mod - 358.200000

A full log can be found at http://www.dynare.org/testsuite/R2016b/master/matlab Run on Matlab R2016b, Linux sedna 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19) x86_64 GNU/Linux _______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev

-- Michel Juillard _______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev

The plan is to run it on an older version once a week. My ideal was to run it on the oldest version of Matlab that we support but I am having trouble installing it on Debian Jessie. I'll install the oldest version I can and run it there.

Sent from my phone

On 19 Nov 2016, at 10:53, Michel Juillard michel.juillard@mjui.fr wrote:

This is a change introduced after R2015a that I'm still using. That's why I couldn't understand what was going on.

I guess that we don't want to use this new feature despite its advantages because it breaks compatibility with Matlab versions that are not so old and what about Octave?

However, because it makes Matlab more permissive, maybe we should still run occasionally the testsuite with an older version of Matlab (once a week?)

Best

Michel

Johannes Pfeifer writes:

...

Hi,

it's not a problem with the testsuite, but with Matlab's convention that you can subtract a conformable vector from a matrix, because Matlab expands the vector to a matrix. Simply try to run

eye(4)-[1 2 3 4]

and you will see that it returns

ans =

0    -2    -3    -4

-1 -1 -3 -4 -1 -2 -2 -4 -1 -2 -3 -3

That's why no error was triggered here. Problems like these are very hard to detect, unless you know the correct answer of the computations and check for them. This might be a next step.

Best, Johannes

-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Samstag, 19. November 2016 10:20 An: List for Dynare developers Betreff: [DynareDev] Testsuite reports erroneous results

I don't undestand how the testsuite can pass test ./kalman_filter_smoother/algo4.mod and several other test cases because of issue https://github.com/DynareTeam/dynare/issues/1339 This test fails on my machine

If the testsuite isn't reliable, we have a major problem

Best

Michel

Dynare Robot writes on 18/11/2016:

...

2f9dc092855bbdb0b3fdd970b6ab551842c5080a diffuse filter: removed special treatment of static variables ================================ DYNARE MAKE CHECK MATLAB RESULTS ================================ | TOTAL: 705 | PASS: 697 | FAIL: 1 | XFAIL: 7 | XPASS: 0 | LIST OF FAILED TESTS: | * arima/mod2a.mod | | LIST OF 10 SLOWEST TESTS: | * optimizers/fs2000_6.mod - 739.560000 | *

observation_trends_and_prefiltering/MCMC/Trend_loglinear_no_prefilter_MC.mod

• 435.730000

...

| *

observation_trends_and_prefiltering/MCMC/Trend_no_prefilter_first_obs_MC.mod

• 434.400000

...

| *

observation_trends_and_prefiltering/MCMC/Trend_loglin_no_prefilt_first_obs_M C.mod - 399.260000

...

| * observation_trends_and_prefiltering/MCMC/Trend_no_prefilter_MC.mod

• 396.370000

...

| *

observation_trends_and_prefiltering/MCMC/Trend_loglin_prefilt_first_obs_MC.m od - 376.320000

...

| * observation_trends_and_prefiltering/MCMC/Trend_prefilter_MC.mod -

372.550000

...

| * ep/rbc_mc.mod - 371.740000 | * gsa/ls2003.mod - 370.280000 | * estimation/TaRB/fs2000_tarb.mod - 358.200000

A full log can be found at http://www.dynare.org/testsuite/R2016b/master/matlab Run on Matlab R2016b, Linux sedna 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19) x86_64 GNU/Linux _______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev

-- Michel Juillard _______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev

Hi Stéphane, good to know. But in the case referenced by Michel it was simply a bug that was hidden because Matlab now implicitly uses bsxfun in the background instead of providing an error regarding non-conformable arrays. And yes, we should not rely on this behavior, but use its expressive version. Best, Johannes

-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Stéphane Adjemain Gesendet: Samstag, 19. November 2016 18:14 An: List for Dynare developers Betreff: Re: [DynareDev] Testsuite reports erroneous results

I use this kind o operations a lot, but with bsxfun which is more general and safer because explicit (also it was introduced in R2007a and is available in Octave). We should use this everywhere, ie replace

eye(4) - [1 2 3 4]

by

bsxfun(@minus, eye(4), [1 2 3 4])

I suppose that, under the hood, matlab calls bsxfun in the first case.

Best, Stéphane

On 19/11/2016 10:53, Michel Juillard wrote:

This is a change introduced after R2015a that I'm still using. That's why I couldn't understand what was going on.

I guess that we don't want to use this new feature despite its advantages because it breaks compatibility with Matlab versions that are not so old and what about Octave?

However, because it makes Matlab more permissive, maybe we should still run occasionally the testsuite with an older version of Matlab (once a week?)

Best

Michel

Johannes Pfeifer writes:

...

Hi,

it's not a problem with the testsuite, but with Matlab's convention that you can subtract a conformable vector from a matrix, because Matlab expands the vector to a matrix. Simply try to run

eye(4)-[1 2 3 4]

and you will see that it returns

ans =

 0    -2    -3    -4
-1    -1    -3    -4
-1    -2    -2    -4
-1    -2    -3    -3

That's why no error was triggered here. Problems like these are very hard to detect, unless you know the correct answer of the computations and check for them. This might be a next step.

Best, Johannes

-----Ursprüngliche Nachricht----- Von: Dev [mailto:dev-bounces@dynare.org] Im Auftrag von Michel Juillard Gesendet: Samstag, 19. November 2016 10:20 An: List for Dynare developers Betreff: [DynareDev] Testsuite reports erroneous results

I don't undestand how the testsuite can pass test ./kalman_filter_smoother/algo4.mod and several other test cases because of issue https://github.com/DynareTeam/dynare/issues/1339 This test fails on my machine

If the testsuite isn't reliable, we have a major problem

Best

Michel

Dynare Robot writes on 18/11/2016:

...

2f9dc092855bbdb0b3fdd970b6ab551842c5080a diffuse filter: removed special treatment of static variables ================================ DYNARE MAKE CHECK MATLAB RESULTS ================================ | TOTAL: 705 | PASS: 697 | FAIL: 1 | XFAIL: 7 | XPASS: 0 | LIST OF FAILED TESTS: | * arima/mod2a.mod | | LIST OF 10 SLOWEST TESTS: | * optimizers/fs2000_6.mod - 739.560000 | *

observation_trends_and_prefiltering/MCMC/Trend_loglinear_no_prefilter _MC.mod

• 435.730000

...

| *

observation_trends_and_prefiltering/MCMC/Trend_no_prefilter_first_obs _MC.mod

• 434.400000

...

| *

observation_trends_and_prefiltering/MCMC/Trend_loglin_no_prefilt_firs t_obs_M C.mod - 399.260000

...

| * | observation_trends_and_prefiltering/MCMC/Trend_no_prefilter_MC.mod

• 396.370000

...

| *

observation_trends_and_prefiltering/MCMC/Trend_loglin_prefilt_first_o bs_MC.m od - 376.320000

...

| * | observation_trends_and_prefiltering/MCMC/Trend_prefilter_MC.mod -

372.550000

...

| * ep/rbc_mc.mod - 371.740000 | * gsa/ls2003.mod - 370.280000 | * estimation/TaRB/fs2000_tarb.mod - 358.200000

A full log can be found at http://www.dynare.org/testsuite/R2016b/master/matlab Run on Matlab R2016b, Linux sedna 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19) x86_64 GNU/Linux _______________________________________________ Dev mailing list Dev@dynare.org https://www.dynare.org/cgi-bin/mailman/listinfo/dev

-- Stéphane Adjemian DynareTeam