17-06-2021 дата публикации
Номер: US20210182661A1
Training and enhancement of neural network models, such as from private data, are described. A slave device receives a version of a neural network model from a master. The slave accesses a local and/or private data source and uses the data to perform optimization of the neural network model. This can be done such as by computing gradients or performing knowledge distillation to locally train an enhanced second version of the model. The slave sends the gradients or enhanced neural network model to a master. The master may use the gradient or second version of the model to improve a master model. 1. A method of training a neural network model , the method comprising:receiving, at a slave device, first configuration data for the neural network model from a master device, the master device being remote from the slave device, the master device including a first version of the neural network model;instantiating, at the slave device, a second version of the neural network model using the first configuration data;training, at the slave device, the second version of the neural network model using data from a first data source, the first data source being inaccessible by the master device; andreceiving, at the master device, second configuration data for the neural network model, from the slave device, based on the trained second version of the neural network model,wherein the master device is configured to use the second configuration data to update parameters for the first version of the neural network model.2. The method of further comprising:instantiating, at the slave device, the second version of the neural network model as a student model;instantiating, at the slave device, the first version of the neural network as a teacher model;using, at the slave device, the teacher model to train the student model; andgenerating the second configuration data to include parameters for the trained student model.3. The method of claim 2 , wherein the first configuration data ...
Подробнее
