Deep learning that makes possible to solve complex non-linear problems on computers is now utilized in a wide range of research fields. For practical applications of deep learning, its energy efficiency is becoming the most critical issue, which is the calculation amount per power consumption. To tackle this issue, researching either of algorithmic improvement or device development is not enough because these two components have a complementary relationship with each other. In our laboratory, we are developing both hardware-oriented deep learning algorithms and dedicated hardware devices for them.