Recoil:Determinism in Engine

The synchronised code in the Recoil Engine is deterministic. Therefore certain considerations must be observed when working in these areas of code.

General Programming

There are a few programming practices that should be observed in the engine simulation code to help avoid introducing desyncs.

Use the Recoil provided RNG function.
Use only stable sorting algorithms i.e. a sorting algorithm that maintains the relative order of the items with equal sort keys.
Avoid relying on sort of order of strings - different locales can have different sort orders.
Do not call functions as arguments to other functions - the order of the calls are not guaranteed. Instead, call them before the function and store the return value in a local variable.

This example will risk a desync:

DoSomething(GetRNG(), GetRNG());

Instead do this:

int firstNumber = GetRNG();
int secondNumber = GetRNG();
DoSomething(firstNumber, secondNumber);

Multi-threading

Multi-threading parts of the engine may offer the opportunity for better performance; however, there are several situations that can cause the engine to desync. So avoid the following:

Do not trigger script call-backs during an MT section. Scripts, whether COB, BOS, Lua, or LUS, are not thread-safe. Collect the calls that need to be made and issue them in a proceeding single-threaded section.
Synced parameters must not be modified in an MT section because they update the sync checksum on update. Store the new value in a separate parameter and apply the new value in a proceeding single-threaded section.
When a parameter is updated when visited per job, the parameter needs to be expanded in an array with a number of entries equal to ThreadPool::MAX_THREADS. This allows each thread to tracking visiting the parameter without interfering with other threads' visits by using an index == thread number.
Pay careful attention to whether a job output can influence an input into another job. For example, don't move units if the jobs also check the unit's position.
Beware of lists resizing and potentially reordering their elements as a result.

SSE Instructions

Do not use the reciprocal functions: Intel specified a maximum error guarantee, which allowed them to use make adjustments depending on the needs of each processor family, but that can result in different results at the bit level.

As a result, do not use the following SSE functions:

_mm_rcp_ps()

_mm_rcp_ss()

_mm_rsqrt_ps()

_mm_rsqrt_ss()

FPU (x87)

The engine uses the strepflop library to setup all FPU-related CPU-bound hardware to ensure that regardless of whether a compiler has used SSE or FPU, general floating point calculations will conform to IEEE754. This is important because most third party libraries are pre-compiled to use the FPU rather than SSE.

The engine also checks multiple times in each sim frame that the FPU options are correct.

Hints for debugging a desync

https://springrts.com/wiki/Debugging_sync_errors

Imported verbatim from the RecoilEngine wiki on GitHub (beyond-all-reason/RecoilEngine). See the source repository for full page history and authorship.