Disclosed are a method and apparatus for generating an ultra-light binary neural network which may be used by an edge device, such as a mobile terminal. A method of re-configuring a neural network includes obtaining a neural network model on which training for inference has been completed, generating a neural network model having a structure identical with the neural network model on which the training has been completed, performing sequential binarization on an input layer and filter of the generated neural network model for each layer, and storing the binarized neural network model. The method may further include providing the binarized neural network model to a mobile terminal.