Zorro version 3.11 is now available on the Download page under 'release candidate'. The full list of the new features can be found on https://zorro-project.com/manual/en/news.htm. This version is the basis for the RangerZ strategy.

This release candidate will become the official release when no bugs are found in the next time. Please test everything and report any issues here! As usual, anyone who finds a new and serious bug in the release candidate will get a free Zorro S subscription or support extension.

Hi JCL!

I'm testing the Monte Carlo Reality Check in the Evaluation Shell with
Workshop6c and would like to confirm whether the behavior below is expected.

My MRC configuration is essentially:

Command,MRC
Method,Train
Cycles,10

Workshop6c uses WFO training and NumCores = -2.

I repeated the same Step 5 MRC command in separate Zorro processes and observed:

First experiment:
- 10 completed runs
- 8 runs: P-Value 0.0%, Result is significant
- 2 runs: P-Value 10.0%, Result insignificant
- no non-empty Errors.txt

Second experiment:
- 20 completed runs
- 17 runs: P-Value 0.0%, Result is significant
- 3 runs: P-Value 10.0%, Result insignificant
- no non-empty Errors.txt

I did not observe intermediate p-values. All results were either 0.0% or
10.0%.

After inspecting the Evaluation Shell MRC path, my current interpretation is
that the p-value is incremented in steps equivalent to:

100 / NumTotalCycles

Therefore, with Cycles=10, the p-value resolution is 10 percentage points:

- zero randomized exceedances -> P-Value 0.0%
- one randomized exceedance -> P-Value 10.0%

At a 5% significance threshold, one randomized exceedance therefore changes
the classification from significant to insignificant. This seems consistent
with ordinary Monte Carlo sampling variability rather than necessarily an
Evaluation Shell defect.

Could you please confirm the following?

1. Is alternating between 0.0% and 10.0% expected with Cycles=10?

2. Does the Cycles value include the original/reference cycle, or does it
represent only randomized cycles?

3. Is Cycles=10 intended only as a quick diagnostic setting?

4. What number of cycles is recommended for evaluating a 5% significance
threshold? Would 100 cycles be a reasonable minimum?

5. With Method=Train, is it expected that the MRC retraining overwrites the
normal WFO parameter and factor files, such as Workshop6c.par,
Workshop6c_1.par, etc.?

6. Is there a supported way to set or record the random seed used by the MRC?

7. Can NumCores=-2 change the random sequence or reproducibility of the MRC?
Should a reproducibility diagnostic use a single core?

Each invocation starts a new Zorro process. The strategy, data, MRC
configuration, and Step 1-4 prerequisite artifacts remain the same, but
Method=Train regenerates the normal WFO outputs during every MRC run.

I am using a locally adapted headless dispatcher around the Evaluation Shell.

Thanks.