Integrated sensing and communication (ISAC) has evolved into an influential technique to ameliorate energy and spectrum scarcity via co-designing these two functionalities. However, the target can be a potential eavesdropper aiming at wiretapping the information transmitted to the communication user. This paper studies a semi-passive intelligent reflecting surface (IRS) enabled ISAC system, where the IRS is employed to assist the secure communication and simultaneously perform the target sensing based on the echo signals received by the dedicated sensor at the IRS. Specifically, we model two types of targets, namely point targets and extended targets. The direction-of-arrival (DoA) of the former and the complete target response matrix of the latter should be estimated. Under this configuration, we derive the Cramér-Rao bound (CRB) as the performance metric of target estimation. To achieve an optimal performance trade-off, we formulate a weighted optimization problem that balances maximizing the secrecy rate and minimizing the CRB, via jointly optimizing the transmit beamforming and the phase shifts of IRS. Then, we employ the alternating optimization, successive convex approximation and semi-definite relaxation to tackle the proposed non-convex problems for the two target cases. Simulation results show the effectiveness of the proposed schemes compared with benchmarks.