Image denoising has become a crucial task in medical Ultrasound (US) imaging due to the presence of speckle or multiplicative noise and additive Gaussian noise. Recently, there are several denoising techniques such as AWT+JB Filter, AFSWM Filter, MPBLOW Filter have been proposed to remove the speckle noise. However, these denoising techniques were found to remove noise along with essential parts of the actual image data which is referred as over-filtering. Thereby, it reduces the accuracy of the recognition process. In this paper, a new hybrid filter technique is proposed by combining Anisotropic Diffusion (AD) with Butterworth band pass filter to overcome the over-filtering of the image. In addition, the performance of the proposed hybrid filter and its design parameters are enhanced by using Particle Swarm Optimization (PSO) algorithm. The simulation results show that the proposed filtering technique achieves a better denoising performance on comparisons with other filtering techniques in terms of PSNR, SNR, SSIM, and EPI. Moreover, the results validate that the proposed filtering technique using PSO achieves effective performance than using Harmony Search Algorithm (HSA) and other filtering techniques.