Rao-Blackwellized particle filter(RBPF) based track-before-detect(TBD) is proposed for detection and tracking of a variable number of weak targets which are moving in infrared image with low Signal to Noise Ratio (SNR). For multiple filters tracking multiple targets a constrained initialization function of new target is constructed by the states of the present targets. With RBPF the state vector can be partitioned as the linear state variable and the nonlinear state variable which can be respectively computed by Kalman filter and particle filter. The simulation results of the infrared image sequences show that constrained initialization can avoid the inference of present targets. RBPF-TBD can reduce the state’s estimation error and improve the detection probability . Efficiency of our algorithm is validated by the experiment in different space position.