INCREMENTAL PRECISION NETWORKS USING RESIDUAL INFERENCE AND FINE-GRAIN QUANTIZATION

Abstract

One embodiment provides for a computer-readable medium storing instructions that cause one or more processors to perform operations comprising determining a per-layer scale factor to apply to tensor data associated with layers of a neural network model and converting the tensor data to converted tensor data. The tensor data may be converted from a floating point datatype to a second datatype that is an 8-bit datatype. The instructions further cause the one or more processors to generate an output tensor based on the converted tensor data and the per-layer scale factor.

IPC Classes  ?

• G06N 3/08 - Learning methods
• G06T 15/00 - 3D [Three Dimensional] image rendering
• G06N 5/04 - Inference or reasoning models
• G06N 3/04 - Architecture, e.g. interconnection topology
• G06F 9/46 - Multiprogramming arrangements
• G06N 3/063 - Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons using electronic means