Invertible Rescaling Net (IRN) modeled the downscaling and up-scaling process using Invertible Neural Networks (INN) instead of upscaling to the traditional Singleimage super resolution (SISR) method. As a result, it showed significantly improved performance than the previous method. However, apart from its high performance, IRN requires a lot of computation. hence, to improve this, we replace the existing dense block with Pixel Attention Distillation Block (PADB). In addition, we use Charbonnier loss instead of Mean Absolute Error (MAE) for the existing reconstruction loss. Through these improvements, we trade off the high performance and speed of the existing architecture and achieve higher performance than the lightweight SR model using the conventional method. In addition, by improving the perceptual loss and adversarial loss. we achieve perceptually satisfactory results than the model using the IRN+ method.