CAPE-OPEN

In this case the easiest way to debug is to make sure you are running the Debug version of the DLL (regsvr32 for the Debug DLL), then start ProII, then from Visual Studio, choose Attach to Process from the Debug menu. Then select the ProII process - make sure to enable Native Debugging.

The missing value for a real number in CAPE-OPEN is IEEE NaN. This is specified in the M&T guidelines.

This complies with e.g. _isnan(), std::isnan() and std::numeric<double>::quiet_nan()

So NaN is the equivalent of RMISS.

Ok, what does that come down to? As I read your answer, it should work. But it doesn't.
Is there anything I can do?

A possible solution to your problem would be to not allow missing values on your parameter. Make sure that all parameters have a default and initial value, and if the simulator tries to set a value that is missing or out of bounds, raise an error. Implement Validate on your parameter accordingly.

Another potential solution is to have a GUI that allows the user to determine which are the specifications, and adapt the parameter collection accordingly. Nearly all COUSCOUS unit operations work like that.

What is then the difference to the status quo? It is already a GUI with fields to be filled in for each parameter.

Instead of having e.g. length = not specified, volume is specified, or volume is specified and length is not specified, you would ask the user in the dialog: specify length or volume? In one case length becomes an input and volume an output, in the other case vice versa. In no case are input parameter unspecified in this scenario.