Training of oscillatory neural networks

CLAIMS What is claimed is: 1. A computer-implemented method for neural network training comprising: encoding, by one or more computer processors, data defining a training sample as a vector of time delays in a network input signal; supplying, by one or more computer processors, the input signal to an oscillatory neural network comprising at least one network layer in which a plurality of electronic oscillators, interconnected via programmable coupling elements storing respective network weights, generate oscillatory signals at time delays dependent on the input signal to propagate the input signal from an input to an output of that layer, the oscillatory neural network being adapted to provide a network output signal dependent substantially linearly on phase of oscillatory signals in the last said layer of the oscillatory neural network; calculating, by one or more computer processors, a network error dependent on said output signal and a desired output for the training sample; calculating, by one or more computer processors, updates for respective network weights by backpropagation of said error such that weight-updates for a network layer are dependent on a vector of time delays at the input to that layer and the calculated error at the output of that layer; and programming, by one or more computer processors, said coupling elements to update the stored weights in accordance with said weight-updates. 2. The computer-implemented method of claim 1, wherein said network output signal is dependent on phase difference between oscillatory signals in a last layer and a reference signal. 3. The computer-implemented method of claim 1, wherein said network error comprises a derivative of a cost function dependent on said output signal and said desired output for the training sample. 4. The computer-implemented method of claim 3, wherein said cost function is dependent on difference between said desired output for the training sample and a predetermined non-linear function of said output signal.

5. The computer-implemented method of claim 4, wherein said non-linear function comprises a ReLU function and wherein the network output signal is supplied to an activation stage, comprising a set of diodes, for applying the ReLU function to the network output signal

6. The computer-implemented method of claim 1, wherein the oscillatory neural network comprises a plurality of layers

7. The computer-implemented method of claim 6, wherein the output of each layer is connected to an activation stage for applying a predetermined non-linear function to an output signal of that layer

8. The computer-implemented method of claim 7, wherein said non-linear function comprises a ReLU function and wherein said activation stage comprises a set of diodes

9. The computer-implemented method of claim 1, wherein: each oscillator comprises a field-effect transistor; and each time delay encoded in the input signal determines timing of a signal applied to a gate of a said transistor in the first network layer to initiate generation of an oscillatory signal by that transistor

10. A neural network system comprising: an encoder adapted to encode data defining a training sample as a vector of time delays in a network input signal; an oscillatory neural network for receiving the input signal, the network comprising at least one network layer in which a plurality of electronic oscillators, interconnected via programmable coupling elements storing respective network weights, generate oscillatory signals at time delays dependent on the input signal to propagate the input signal from an input to an output of that layer, the oscillatory neural network being adapted to provide a network output signal dependent substantially linearly on phase of oscillatory signals in the last said layer of the oscillatory neural network; a system controller adapted to calculate a network error dependent on said output signal and a desired output for the training sample, to calculate updates for respective network weights by backpropagation of said error such that weight-updates for a network layer are dependent on a vector of time delays at the input to that layer and the calculated error at the output of that layer, and to program said coupling elements to update the stored weights in accordance with said weight-updates

11. The neural network system of claim 10, wherein said network output signal is dependent on phase difference between oscillatory signals in a last layer and a reference signal

12. The neural network system of claim 10, wherein said network error comprises a derivative of a cost function dependent on difference between said desired output for the training sample and a predetermined non-linear function of said output signal

13. The neural network system of claim 12, wherein said non-linear function comprises a ReLU function and wherein the network output signal is supplied to an activation stage, comprising a set of diodes, for applying the ReLU function to the network output signal

14. The neural network system of claim 10, wherein the oscillatory neural network comprises a plurality of layers

15. The neural network system of claim 14, wherein the output of each layer is connected to an activation stage for applying a predetermined non-linear function to an output signal of that layer

16. The neural network system of claim 15, wherein said non-linear function comprises a ReLU function and wherein said activation stage comprises a set of diodes

17. The neural network system of claim 10, wherein each oscillator comprises a field- effect transistor and the network is adapted such that each time delay encoded in the input signal determines timing of a signal applied to a gate of a said transistor in the first network layer to initiate generation of an oscillatory signal by that transistor .

18. The neural network system of claim 10, wherein the coupling elements comprise resistive memory devices.

19. The neural network system of claim 10, wherein each said oscillator comprises a relaxation oscillator .

20. A computer program product for training an oscillatory neural network, the computer program product comprising a computer readable storage medium having program instructions embodied therein, the program instructions being executable by a processing apparatus to cause the processing apparatus to: encode data defining a training sample as a vector of time delays in a network input signal; supply the input signal to an oscillatory neural network comprising at least one network layer in which a plurality of electronic oscillators, interconnected via programmable coupling elements storing respective network weights, generate oscillatory signals at time delays dependent on the input signal to propagate the input signal from an input to an output of that layer, the oscillatory neural network being adapted to provide a network output signal dependent substantially linearly on phase of oscillatory signals in the last said layer of the oscillatory neural network; calculate a network error dependent on said output signal and a desired output for the training sample; calculate updates for respective network weights by backpropagation of said error such that weight-updates for a network layer are dependent on a vector of time delays at the input to that layer and the calculated error at the output of that layer; and program said coupling elements to update the stored weights in accordance with said weight-updates.