Somewhat similar to whitelisting on a host pc, you can implement software checking on the real-time controller of the RIO device. The strategy is to compute the checksum of key system files on the real-time controller and check them at startup against the stored values.
A meaningful implementation would be to have the checksums computed and stored either on portable media, such as a USB drive, or on the host pc. The first step of the real-time application can be to compute the checksum of key files such as ni-rt.ini and return the value to the host pc for verification or to check the value against the value stored on removable media such as a USB drive plugged into the controller. If the checksum differs from the value previously computed, you know that something on the real-time controller has been modified and can investigate the change further and take action, such as to reformat the real-time controller.
For computing a checksum on key files, consider using the File Checksum VI, which is a built-in function which can run on LabVIEW Real-Time. For LabVIEW versions prior to LabVIEW 2020, consider using MD5Checksum File.VI.