q-Neurons: Neuron Activations Based on Stochastic Jackson’s Derivative Operators.

Researchers

Journal

Modalities

Models

Abstract

We propose a new generic type of artificial neurons called q-neurons. A q-neuron is a stochastic neuron with its activation function relying on Jackson’s discrete q-derivative for a stochastic parameter q. We show how to generalize neural network architectures with q-neurons and demonstrate the scalability and ease of implementation of q-neurons into legacy deep learning frameworks. We report experimental results that consistently improve performance over state-of-the-art standard activation functions, both on training and test loss functions.