When designing a deep learning model, whenever a decision must be made, the answer is simple: multiply your data by a weight matrix. For example, in an LSTM, need to forget some data? Just multiply by a weight matrix to get the forget gate. The system will learn what you meant.