3D imaging based on single-photon avalanche diode (SPAD) arrays is of interest in many applications. Limited by the small array size and low fill-factor of the available SPAD array, it is difficult for SPAD array-based lidars to achieve high-resolution and high-quality 3D images currently. Here, we propose a sub-pixel continuously scanning panoramic 3D imaging method. This method uses sub-pixel scanning to increase spatial resolution and employs accumulating photon counts at the different pixel of the array for the same target point to improve 3D image quality. We have established an experimental system based on a 32x32 SPADarray and a diffraction optical element (DOE) with 32 x 32 beamlet. By sub-pixel scanning for a target 3 m away and accumulating photon counts, we demonstrated that the spatial resolution has been improved from 3 mm to 0.33 mm and the 3D image quality has been improved evidently.