Artificial Intelligence

   

Dynamics of Feed Forward Induced Interference Training

Authors: Shirui Tang

Preceptron model updating with back propagation has become the routine of deep learning. Continu-ous feed forward procedure is required in order for backward propagate to function properly. Doubt-ing the underlying physical interpretation on transformer based models such as GPT brought aboutby the routine explaination, a new method of training is proposed in order to keep self-consistencyof the physics. By treating the GPT model as a space-time diagram, and then trace the worldlinesof signals, identifing the possible paths of signals in order fot a self-attention event to occure. Witha slight modification, self-attention can be viewed as an ising model interaction, which enables thegoal to be designed as energy of system. Target is treated as an external magnetic field inducing sig-nals modeled as magnetic dipoles. A probability network is designed to pilot input signals travellingat constant speed through different routes. A rule of updating the probabilities is designed in orderto form constructive interference at target locations so that instantaneous energy can be maximised.Experiment is conducted on a 4-class classification problem extracted from MNIST. The results ex-hibit interesting but expected behavours, which do not exist in a bp updated network, but more likelearning in a real human, especially in the few-shot scenario.

Comments: 12 Pages.

Download: PDF

Submission history

[v1] 2020-08-22 05:30:26

Unique-IP document downloads: 216 times

Vixra.org is a pre-print repository rather than a journal. Articles hosted may not yet have been verified by peer-review and should be treated as preliminary. In particular, anything that appears to include financial or legal advice or proposed medical treatments should be treated with due caution. Vixra.org will not be responsible for any consequences of actions that result from any form of use of any documents on this website.

Add your own feedback and questions here:
You are equally welcome to be positive or negative about any paper but please be polite. If you are being critical you must mention at least one specific error, otherwise your comment will be deleted as unhelpful.

comments powered by Disqus