A new method to the correction and compensation of dynamic error of the optical encoder was presented

to which the neural network and the digital signal process technology were both applied. In the paper

the modeling method based on the Radial Basis Function（RBF）was set up

in which the output was test value of the high precision instrument and the input was angle value of sample points. According to the inhibit condition between the test value and the output of the network

adjusting power factor formula and the center and width of the radial basis function to make the model have a good learning ability and generalization ability

the error curve was recovered. The relationship between the sampled angles and the errors can be determined by training the neural network. After the sampled angles were measured

the unknown error of the encoder can be calculated via a trained neural network even if the error was nonlinear. The characteristic of this method was that，it only use the test data and there were no needs for knowledge of the reason for the error and the experience about the error form. Because the model was set up with too many parameters and lots of operation

the high-speed controller was needed. The TMS320LF2407 was chosen in the design. The device offers the enhanced architectural design of the CPU for low-cost

low-power

and high-performance processing capabilities. Also it offers increased processing performance (30 MIPS) and a higher level of peripheral integration More than that

the LF2407 incorporates one 32K 16-bit Flash EEPROM module in program space .The LF2407 Flash does not require a dedicated state machine

because the algorithms for programming and erasing the Flash are executed by the DSP core. The parameters of the model can be reserved on time by using the advantage of “in-target” reprogram ability of the flash

and the error correction for the system can be achieved without disassembling the system. Flash makes the update of program，system maintenance and upgrade easier as well. The testing results show that the precision of system error by this method is enhanced 3-5 times even advanced by roughly an order of magnitude. And not only the error of each network point but also the interpolated network point can be corrected automatically .The practical application proves that the precision of measuring system is improved greatly by using the RBF model as error compensation

and the nonlinear effect on the system is reduced.