(back to discussion topics)

  Accuracy and Precision in MOPAC

There are three quantities that need to be carefully described in order to understand the significance of numerical values generated by MOPAC: Accuracy, reproducibility and precision.  First, some definitions:

Accuracy:  A comparison of the results obtained by calculation with the values of reference data.  This is usually expressed in terms of Average Unsigned Errors for various quantities, See accuracy for PM7 for examples and details.

Reproducibility: A method such as PM7 consists of a set of approximations and a set of parameters.  Such methods can be used in programs such as MOPAC, that can be run on different operating systems, e.g., Windows, Linux, and MAC, that run in various architectures, i.e., 32 or 64 bit, using variable numbers of processors (threads), typically 1 to 12 threads.  Results obtained  from running the same system using different programs, OS, architectures, and number of threads, should be the same. In addition, unless a method is very deliberately changed, results obtained at one time should be reproducible at any later date.

Precision: The number of significant figures in a printed quantity. 

Accuracy

 The recent semiempirical methods (PM6 and PM7) in MOPAC are only good for modeling chemical systems, i.e., for predicting the geometries of various species and their heats of formation.  For non-chemical properties, such as ionization potentials and dipole moments, or electronics in general, these methods are of low accuracy.

Reproducibility

In principle, the results of calculations run using a specific method on a well-defined system should be independent of all variables such as software, operating system, architecture, etc.  In practice, this is normally does not happen - there are small differences.  Despite the fact that all versions of MOPAC use the same software, and all versions are compiled using the same compiler (the Intel Fortran compiler), there are small differences in the results.  These differences can be attributed to a small number of specific factors:

(A) Finite convergence criteria are used.  In the SCF calculation, the optimization of the wavefunction is terminated when a pre-set criterion is satisfied.  In geometry optimization, the process is stopped when the gradients fall below a pre-set limit. These criteria were selected to minimize run-times while not significantly compromising the results.  If better convergence is needed, the criteria can be tightened, e.g., by adding RELSCF=0.1 (for the SCF criterion) or GNORM=0.5 (for geometries), or, in general, by adding PRECISE.

(B) Although the same software is used on all platforms, differences in the operating systems can alter the math used in subtle ways.  All arithmetic is carried out to 16 digit precision, but the last digit can have an uncertainty of ±1.  This uncertainty can become greater as more and more math is carried out, and in some cases can result in a degradation from 16 digit precision to five or even fewer digits.  In most cases, this loss of numerical precision can, perhaps surprisingly, be corrected by tightening up the convergence criteria.  With the default criteria, the path from the starting function (electronic or geometric) to the function that satisfies the criteria can differ from one machine to another.  But if the criteria are tightened up, the final functions will also be more tightly defined, and differences in the paths taken to reach the optimum point will become less and less important.

(C) Even on one machine, if the same job is run several times, small differences can be detected.  This effect occurs only on multi-processor machines, and only when multi-thread calculations (the default) are run.  In this case, the differences originate in the changing sequence of operations that can occur when multi-processing is done.  This type of error can be eliminated by using only one thread, using THREADS=1, or by limiting processes to one core.

(D) There are two very different methods in MOPAC for solving the self-consistent equations: (1) Conventional SCF methods, that use matrix algebra, and scale as the cube of the size of the system. (1) A Localized Molecular Orbital method, MOZYME, that uses Lewis structures to general LMOs.  SCF calculations run using MOZYME scale as the size of system to the power 1.4, i.e., almost linear scaling. MOZYME was written to allow protein systems (protein plus water plus hetero groups, e.g., substrates) to be run in a reasonable time.  Currently, MOZYME  is limited to RHF systems.  Results calculated using conventional methods and using MOZYME should be the same.  In practice,  because of different criteria, there are small differences.  These differences can be made arbitrarily small by tightening criteria (under keyword control) but if that is done, the compute effort would increase to the point that MOZYME would take the same time as conventional.  In most cases where MOZYME gives a significantly different result from conventional, and the MOZYME heat of formation is greater than the conventional, MOZYME has converged on an electronic excited state.  In the remaining cases, the conventional SCF has converged on an electronic excited state/

Precision

In this context, precision is the number of significant digits in a quantity that is printed.  The internal precision of the calculation is not readily accessible to the user, and is therefore essentially irrelevant.  In most cases, the precision used in the output and related files is within two or three digits of the internal precision of the calculation. If higher or lower precision is needed, use AUX to output results to the required precision.

Semiempirical methods have limited accuracy, so although the AUE for DHf might be 4 or 5 kcal/mol, the precision of the results will typically be 10-6 kcal/mol.  For most work, the precision of DHf should be truncated to 0.1 kcal/mol.