Hossein Hosseini was a research intern at NEC Laboratories America, Inc. while studying at University of Washington.

Posts

Learning from Rules Performs as Implicit Regularization

/in /by

In this paper, we study the generalization performance of deep neural networks in learning problems where the given task is governed by a set of rules. We consider two settings of supervised learning and rule-based learning. In supervised learning, the network is trained with pairs of inputs and the corresponding solutions that satisfy the problem constraints. In rule-based learning, the constraints are encoded into a neural network module that is applied on the output of the solver network. In this approach, instead of training with any actual solutions of the problem, the model will be trained to explicitly satisfy the constraints. We perform the experiments on two problems of solving a system of nonlinear equations and solving Sudoku puzzles. Our experimental results show that, compared to supervised approach, rule-based learning results in higher training error, but significantly lower validation error, especially when training data is small, thus performing as an implicit regularization.

Read more

Battery Degradation Temporal Modeling Using LSTM Networks

/in /by

Accurate modeling of battery capacity degradation is an important component for both battery manufacturers and energy management systems. In this paper, we develop a battery degradation model using deep learning algorithms. The model is trained with the real data collected from battery storage solutions installed and operated for behind-the-meter customers. In the dataset, battery operation data are recorded at a small scale (five minutes) and battery capacity is measured at every six months. In order to improve the training performance, we apply two preprocessing techniques, namely subsampling and feature extraction on operation data, and also interpolating between capacity measurements at times for which battery operation features are available. We integrate both cyclic and calendar aging processes in a unified framework by extracting the corresponding features from operation data. The proposed model uses LSTM units followed by a fully-connected network to process weekly battery operation features and predicts the capacity degradation. The experimental results show that our method can accurately predict the capacity fading and significantly outperforms baseline models including persistence and autoregressive (AR) models.

Read more

Conditioning Neural Networks: A Case Study of Electrical Load Forecasting

/in /by

Machine learning tasks typically involve minimizing a loss function that measures the distance of the model output and the ground-truth. In some applications, in addition to the usual loss function, the output must also satisfy certain requirements for further processing. We call such requirements model conditioning. We investigate cases where the conditioner is not differentiable or cannot be expressed in closed form and, hence, cannot be directly included in the loss function of the machine learning model. We propose to replace the conditioner with a learned dummy model which is applied on the output of the main model. The entire model, composed of the main and dummy models, is trained end-to-end. Throughout training, the dummy model learns to approximate the conditioner and, thus, forces the main model to generate outputs that satisfy the specified requirements. We demonstrate our approach on a use-case of demand charge-aware electricity load forecasting. We show that jointly minimizing the error in forecast load and its demand charge threshold results in significant improvement to existing load forecast methods.

Read more