An object of this invention is to effectively reduce a luminance flicker caused by noise while suppressing an adverse effect such as an afterimage and a decrease in the resolution upon smoothing as much as possible even when the S/N ratio of an image photographed at an increased gain is low. An image signal processing apparatus includes a smoothing circuit (124) to execute smoothing processing of an image signal, a compression encoding circuit (112, 113) to compression-encode the image signal, a setting circuit (126) capable of setting, for the compression encoding circuit, the code amount for each picture type, and a control circuit (116) that controls in accordance with a gain setting value acquired from a camera (102) to change the intensity of smoothing processing and the assignment ratio of code amount for each picture type.